With the ubiquitous presence of digital cameras and camera phones, people capture large numbers of images and videos to mark events important to them. The highlights of such events are then shared online to be accessed by the user's social networks. Large digital media collections accumulated over time contain a wealth of information that can be useful to understand individual users as well as groups of people. Temporal information is valuable for information management and retrieval in general, enhancing search and browsing applications. Analyzing the content and timing of a user's media in a collection that spans several years can yield dates of importance and a view into the user's interests. This knowledge can enable organization of the personal collection, sharing with contacts, as well as personalized and well-timed advertising. For example, if evidence from a user's personal photo collection suggests that he/she regularly takes a vacation during a school break in March, the images in this group can be organized appropriately with links to previous years' vacations. Travel and tourism-related advertising can be targeted to fall within the planning phase of this time period, and the images can be shared with contacts with which the user regularly shares this type of images.
Attempting to identify some of these events using a generic calendar of important dates can detect a limited number of events, and none of the user-specific special dates (e.g. birthdays) can be detected in this manner. Also, this approach makes an assumption that the user actually celebrates the same holidays as the region they are in, when in reality there would need to be a different calendar for each group of people in a diverse population. In addition to differences in calendar due to cultural differences, the location of the user also contributes local events to the calendar e.g. Lilac Festival in Rochester, N.Y., International Balloon Fiesta in Albuquerque, N. Mex. In response to these problems, there has been work in associating users' captured images with their personal calendars (e.g. “Image Annotation Using Personal Calendars as Context”, Gallagher et al, ACM Intl. Conf. on Multimedia 2008). However, notations on personal calendars often relate to appointments and work tasks that are not associated with picture-taking.
There has been work in grouping images into events. U.S. Pat. No. 6,606,411 by Loui and U.S. Pat. No. 6,351,556 by Loui, disclose algorithms for clustering image content by temporal events and sub-events. According to U.S. Pat. No. 6,606,411 events have consistent color distributions, and therefore, these pictures are likely to have been taken with the same backdrop. For each sub-event, a single color and texture representation is computed for all background areas taken together. The above two patents teach how to cluster images and videos in a digital image collection into temporal events and sub-events. The terms “event” and “sub-event” are used in an objective sense to indicate the products of a computer mediated procedure that attempts to match a user's subjective perceptions of specific occurrences (corresponding to events) and divisions of those occurrences (corresponding to sub-events). Another method of automatically organizing images into events is disclosed in U.S. Pat. No. 6,915,011 by Loui et al. The events detected are chronologically ordered in a timeline from earliest to latest.
Using the above methods, a reduction can be made in the amount of browsing required by the user to locate a particular event by viewing representatives of the events along a timeline, instead of each image thumbnail. However, due to the large temporal separation of related events (such as birthdays), these event groups are spaced far apart on the timeline and are not easy to visualize as a group. Therefore, a need exists to detect groups of images that are semantically related to each other but are temporally separated by long time differences.